{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 宽度深度模型/wide and deep model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 介绍\n",
    "\n",
    "在之前的代码里大家看到了如何用tensorflow自带的op来构建灵活的神经网络，这里用tf中的高级接口，用更简单的方式完成wide&deep模型。\n",
    "\n",
    "大家都知道google官方给出的典型wide&deep模型结构如下：\n",
    "![](https://img-blog.csdn.net/20170502135611349)\n",
    "\n",
    "更一般的拼接模型ctr预估结构可以如下：\n",
    "![](https://yxzf.github.io/images/deeplearning/dnn_ctr/embeding.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 导入工具库"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Using TensorFlow version 1.4.0\n",
      "\n",
      "Feature columns are:  ['I1', 'I2', 'I3', 'I4', 'I5', 'I6', 'I7', 'I8', 'I9', 'I10', 'I11', 'I12', 'I13', 'C1', 'C2', 'C3', 'C4', 'C5', 'C6', 'C7', 'C8', 'C9', 'C10', 'C11', 'C12', 'C13', 'C14', 'C15', 'C16', 'C17', 'C18', 'C19', 'C20', 'C21', 'C22', 'C23', 'C24', 'C25', 'C26'] \n",
      "\n",
      "Columns and data as a dict:  {'C19': 'f6a3e43b', 'C18': 'bd17c3da', 'C13': '7203f04e', 'C12': '79507c6b', 'C11': '77212bd7', 'C10': 'ceb10289', 'C17': '8efede7f', 'C16': '49013ffe', 'C15': '2c14c412', 'C14': '07d13a8f', 'I9': 475, 'I8': 17, 'I1': 0, 'I3': 1, 'I2': 127, 'I5': 1683, 'I4': 3, 'I7': 26, 'I6': 19, 'C9': 'a73ee510', 'C8': '0b153874', 'C3': '11c9d79e', 'C2': '8947f767', 'C1': '05db9164', 'C7': '18671b18', 'C6': 'fbad5c96', 'C5': '4cf72387', 'C4': '52a787c8', 'C22': 'ad3062eb', 'C23': 'c7dc6720', 'C20': 'a458ea53', 'C21': '35cd95c9', 'C26': '49d68486', 'C24': '3fdb382b', 'C25': '010f6491', 'I11': 9, 'I10': 0, 'I13': 3, 'I12': 0} \n",
      "\n"
     ]
    }
   ],
   "source": [
    "from __future__ import absolute_import\n",
    "from __future__ import division\n",
    "from __future__ import print_function\n",
    "\n",
    "import time\n",
    "\n",
    "import tensorflow as tf\n",
    "\n",
    "tf.logging.set_verbosity(tf.logging.INFO)\n",
    "print(\"Using TensorFlow version %s\\n\" % (tf.__version__))\n",
    "\n",
    "# 我们这里使用的是criteo数据集，X的部分包括13个连续值列和26个类别型值的列\n",
    "CONTINUOUS_COLUMNS =  [\"I\"+str(i) for i in range(1,14)] # 1-13 inclusive\n",
    "CATEGORICAL_COLUMNS = [\"C\"+str(i) for i in range(1,27)] # 1-26 inclusive\n",
    "# 标签是clicked\n",
    "LABEL_COLUMN = [\"clicked\"]\n",
    "\n",
    "# 训练集由 label列 + 连续值列 + 离散值列 构成\n",
    "TRAIN_DATA_COLUMNS = LABEL_COLUMN + CONTINUOUS_COLUMNS + CATEGORICAL_COLUMNS\n",
    "#TEST_DATA_COLUMNS = CONTINUOUS_COLUMNS + CATEGORICAL_COLUMNS\n",
    "\n",
    "# 特征列就是 连续值列+离散值列\n",
    "FEATURE_COLUMNS = CONTINUOUS_COLUMNS + CATEGORICAL_COLUMNS\n",
    "\n",
    "# 输出一些信息\n",
    "print('Feature columns are: ', FEATURE_COLUMNS, '\\n')\n",
    "\n",
    "# 数据示例\n",
    "sample = [ 0, 127, 1, 3, 1683, 19, 26, 17, 475, 0, 9, 0, 3, \"05db9164\", \"8947f767\", \"11c9d79e\", \"52a787c8\", \"4cf72387\", \"fbad5c96\", \"18671b18\", \"0b153874\", \"a73ee510\", \"ceb10289\", \"77212bd7\", \"79507c6b\", \"7203f04e\", \"07d13a8f\", \"2c14c412\", \"49013ffe\", \"8efede7f\", \"bd17c3da\", \"f6a3e43b\", \"a458ea53\", \"35cd95c9\", \"ad3062eb\", \"c7dc6720\", \"3fdb382b\", \"010f6491\", \"49d68486\"]\n",
    "\n",
    "print('Columns and data as a dict: ', dict(zip(FEATURE_COLUMNS, sample)), '\\n')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 输入文件解析\n",
    "\n",
    "我们把数据送进`Reader`然后从文件里一次读一个batch \n",
    "\n",
    "对`_input_fn()`函数做了特殊的封装处理，使得它更适合不同类型的文件读取\n",
    "\n",
    "注意一下：这里的文件是直接通过tensorflow读取的，我们没有用pandas这种工具，也没有一次性把所有数据读入内存，这样对于非常大规模的数据文件训练，是合理的。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 关于input_fn函数\n",
    "\n",
    "这个函数定义了我们怎么读取数据用于训练和测试。这里的返回结果是一个pair对，第一个元素是列名到具体取值的映射字典，第二个元素是label的序列。\n",
    "\n",
    "抽象一下，大概是这么个东西 `map(column_name => [Tensor of values]) , [Tensor of labels])`\n",
    "\n",
    "举个例子就长这样：\n",
    "\n",
    "    { \n",
    "      'age':            [ 39, 50, 38, 53, 28, … ], \n",
    "      'marital_status': [ 'Married-civ-spouse', 'Never-married', 'Widowed', 'Widowed' … ],\n",
    "       ...\n",
    "      'gender':           ['Male', 'Female', 'Male', 'Male', 'Female',, … ], \n",
    "    } , \n",
    "    [ 1, 1, 0, 1, 1, 1, 0, 0, 0, 0, 1, 1]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### High-level structure of input functions for CSV-style data\n",
    "1. Queue file(s)\n",
    "2. Read a batch of data from the next file\n",
    "3. Create record defaults, generally 0 for continuous values, and \"\" for categorical. You can use named types if you prefer\n",
    "4. Decode the CSV and restructure it to be appropriate for the graph's input format\n",
    "    * `zip()` column headers with the data\n",
    "    * `pop()` off the label column(s)\n",
    "    * Remove/pop any unneeded column(s)\n",
    "    * Run `tf.expand_dims()` on categorical columns\n",
    "    5. Return the pair: `(feature_dict, label_array)`\n",
    "    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "input function configured\n"
     ]
    }
   ],
   "source": [
    "BATCH_SIZE = 2000\n",
    "\n",
    "def generate_input_fn(filename, batch_size=BATCH_SIZE):\n",
    "    def _input_fn():\n",
    "        filename_queue = tf.train.string_input_producer([filename])\n",
    "        reader = tf.TextLineReader()\n",
    "        # 只读batch_size行\n",
    "        key, value = reader.read_up_to(filename_queue, num_records=batch_size)\n",
    "        \n",
    "        # 1个int型的label, 13个连续值, 26个字符串类型\n",
    "        cont_defaults = [ [0] for i in range(1,14) ]\n",
    "        cate_defaults = [ [\" \"] for i in range(1,27) ]\n",
    "        label_defaults = [ [0] ]\n",
    "        column_headers = TRAIN_DATA_COLUMNS\n",
    "        \n",
    "        # 第一列数据是label\n",
    "        record_defaults = label_defaults + cont_defaults + cate_defaults\n",
    "\n",
    "        # 解析读出的csv数据\n",
    "        # 我们要手动把数据和header去zip在一起\n",
    "        columns = tf.decode_csv(\n",
    "            value, record_defaults=record_defaults)\n",
    "        \n",
    "        # 最终是列名到数据张量的映射字典\n",
    "        all_columns = dict(zip(column_headers, columns))\n",
    "        \n",
    "        # 弹出和保存label标签\n",
    "        labels = all_columns.pop(LABEL_COLUMN[0])\n",
    "        \n",
    "        # 其余列就是特征\n",
    "        features = all_columns \n",
    "\n",
    "        # 类别型的列我们要做一个类似one-hot的扩展操作\n",
    "        for feature_name in CATEGORICAL_COLUMNS:\n",
    "            features[feature_name] = tf.expand_dims(features[feature_name], -1)\n",
    "\n",
    "        return features, labels\n",
    "\n",
    "    return _input_fn\n",
    "\n",
    "print('input function configured')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 构建特征列\n",
    "这个部分我们来看一下用tensorflow的高级接口，如何方便地对特征进行处理"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 稀疏列/Sparse Columns\n",
    "我们先构建稀疏列(针对类别型)\n",
    "\n",
    "对于所有类别取值都清楚的我们用`sparse_column_with_keys()`处理\n",
    "\n",
    "对于类别可能比较多，没办法枚举的可以试试用`sparse_column_with_hash_bucket()`处理这个映射"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Wide/Sparse columns configured\n"
     ]
    }
   ],
   "source": [
    "# Sparse base columns.\n",
    "# C1 = tf.contrib.layers.sparse_column_with_hash_bucket('C1', hash_bucket_size=1000)\n",
    "# C2 = tf.contrib.layers.sparse_column_with_hash_bucket('C2', hash_bucket_size=1000)\n",
    "# C3 = tf.contrib.layers.sparse_column_with_hash_bucket('C3', hash_bucket_size=1000)\n",
    "# ...\n",
    "# Cn = tf.contrib.layers.sparse_column_with_hash_bucket('Cn', hash_bucket_size=1000)\n",
    "# wide_columns = [C1, C2, C3, ... , Cn]\n",
    "\n",
    "wide_columns = []\n",
    "for name in CATEGORICAL_COLUMNS:\n",
    "    wide_columns.append(tf.contrib.layers.sparse_column_with_hash_bucket(\n",
    "            name, hash_bucket_size=1000))\n",
    "\n",
    "print('Wide/Sparse columns configured')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 连续值列/Continuous columns\n",
    "通过`real_valued_column()`设定连续值列"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "deep/continuous columns configured\n"
     ]
    }
   ],
   "source": [
    "# Continuous base columns.\n",
    "# I1 = tf.contrib.layers.real_valued_column(\"I1\")\n",
    "# I2 = tf.contrib.layers.real_valued_column(\"I2\")\n",
    "# I3 = tf.contrib.layers.real_valued_column(\"I3\")\n",
    "# ...\n",
    "# In = tf.contrib.layers.real_valued_column(\"In\")\n",
    "# deep_columns = [I1, I2, I3, ... , In]\n",
    "\n",
    "deep_columns = []\n",
    "for name in CONTINUOUS_COLUMNS:\n",
    "    deep_columns.append(tf.contrib.layers.real_valued_column(name))\n",
    "\n",
    "print('deep/continuous columns configured')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 特征工程变换\n",
    "因为这是一份做过脱敏处理的数据，所以我们做下面的2个操作\n",
    " \n",
    "* **分桶/bucketizing** 对连续值离散化和分桶\n",
    "* **生成交叉特征/feature crossing** 对2列或者多列去构建交叉组合特征(注意只有离散的特征才能交叉，所以如果连续值特征要用这个处理，要先离散化) "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Transformations complete\n"
     ]
    }
   ],
   "source": [
    "# No known Transformations. Can add some if desired. \n",
    "# Examples from other datasets are shown below.\n",
    "\n",
    "# age_buckets = tf.contrib.layers.bucketized_column(age,\n",
    "#             boundaries=[ 18, 25, 30, 35, 40, 45, 50, 55, 60, 65 ])\n",
    "# education_occupation = tf.contrib.layers.crossed_column([education, occupation], \n",
    "#                                                         hash_bucket_size=int(1e4))\n",
    "# age_race_occupation = tf.contrib.layers.crossed_column([age_buckets, race, occupation], \n",
    "#                                                        hash_bucket_size=int(1e6))\n",
    "# country_occupation = tf.contrib.layers.crossed_column([native_country, occupation], \n",
    "#                                                       hash_bucket_size=int(1e4))\n",
    "\n",
    "print('Transformations complete')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Group feature columns into 2 objects\n",
    "\n",
    "The wide columns are the sparse, categorical columns that we specified, as well as our hashed, bucket, and feature crossed columns. \n",
    "\n",
    "The deep columns are composed of embedded categorical columns along with the continuous real-valued columns. **Column embeddings** transform a sparse, categorical tensor into a low-dimensional and dense real-valued vector. The embedding values are also trained along with the rest of the model. For more information about embeddings, see the TensorFlow tutorial on [Vector Representations Words](https://www.tensorflow.org/tutorials/word2vec/), or [Word Embedding](https://en.wikipedia.org/wiki/Word_embedding) on Wikipedia.\n",
    "\n",
    "The higher the dimension of the embedding is, the more degrees of freedom the model will have to learn the representations of the features. We are starting with an 8-dimension embedding for simplicity, but later you can come back and increase the dimensionality if you wish.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:The default stddev value of initializer will change from \"1/sqrt(vocab_size)\" to \"1/sqrt(dimension)\" after 2017/02/25.\n",
      "WARNING:tensorflow:The default stddev value of initializer will change from \"1/sqrt(vocab_size)\" to \"1/sqrt(dimension)\" after 2017/02/25.\n",
      "WARNING:tensorflow:The default stddev value of initializer will change from \"1/sqrt(vocab_size)\" to \"1/sqrt(dimension)\" after 2017/02/25.\n",
      "WARNING:tensorflow:The default stddev value of initializer will change from \"1/sqrt(vocab_size)\" to \"1/sqrt(dimension)\" after 2017/02/25.\n",
      "WARNING:tensorflow:The default stddev value of initializer will change from \"1/sqrt(vocab_size)\" to \"1/sqrt(dimension)\" after 2017/02/25.\n",
      "WARNING:tensorflow:The default stddev value of initializer will change from \"1/sqrt(vocab_size)\" to \"1/sqrt(dimension)\" after 2017/02/25.\n",
      "WARNING:tensorflow:The default stddev value of initializer will change from \"1/sqrt(vocab_size)\" to \"1/sqrt(dimension)\" after 2017/02/25.\n",
      "WARNING:tensorflow:The default stddev value of initializer will change from \"1/sqrt(vocab_size)\" to \"1/sqrt(dimension)\" after 2017/02/25.\n",
      "WARNING:tensorflow:The default stddev value of initializer will change from \"1/sqrt(vocab_size)\" to \"1/sqrt(dimension)\" after 2017/02/25.\n",
      "WARNING:tensorflow:The default stddev value of initializer will change from \"1/sqrt(vocab_size)\" to \"1/sqrt(dimension)\" after 2017/02/25.\n",
      "WARNING:tensorflow:The default stddev value of initializer will change from \"1/sqrt(vocab_size)\" to \"1/sqrt(dimension)\" after 2017/02/25.\n",
      "WARNING:tensorflow:The default stddev value of initializer will change from \"1/sqrt(vocab_size)\" to \"1/sqrt(dimension)\" after 2017/02/25.\n",
      "WARNING:tensorflow:The default stddev value of initializer will change from \"1/sqrt(vocab_size)\" to \"1/sqrt(dimension)\" after 2017/02/25.\n",
      "WARNING:tensorflow:The default stddev value of initializer will change from \"1/sqrt(vocab_size)\" to \"1/sqrt(dimension)\" after 2017/02/25.\n",
      "WARNING:tensorflow:The default stddev value of initializer will change from \"1/sqrt(vocab_size)\" to \"1/sqrt(dimension)\" after 2017/02/25.\n",
      "WARNING:tensorflow:The default stddev value of initializer will change from \"1/sqrt(vocab_size)\" to \"1/sqrt(dimension)\" after 2017/02/25.\n",
      "WARNING:tensorflow:The default stddev value of initializer will change from \"1/sqrt(vocab_size)\" to \"1/sqrt(dimension)\" after 2017/02/25.\n",
      "WARNING:tensorflow:The default stddev value of initializer will change from \"1/sqrt(vocab_size)\" to \"1/sqrt(dimension)\" after 2017/02/25.\n",
      "WARNING:tensorflow:The default stddev value of initializer will change from \"1/sqrt(vocab_size)\" to \"1/sqrt(dimension)\" after 2017/02/25.\n",
      "WARNING:tensorflow:The default stddev value of initializer will change from \"1/sqrt(vocab_size)\" to \"1/sqrt(dimension)\" after 2017/02/25.\n",
      "WARNING:tensorflow:The default stddev value of initializer will change from \"1/sqrt(vocab_size)\" to \"1/sqrt(dimension)\" after 2017/02/25.\n",
      "WARNING:tensorflow:The default stddev value of initializer will change from \"1/sqrt(vocab_size)\" to \"1/sqrt(dimension)\" after 2017/02/25.\n",
      "WARNING:tensorflow:The default stddev value of initializer will change from \"1/sqrt(vocab_size)\" to \"1/sqrt(dimension)\" after 2017/02/25.\n",
      "WARNING:tensorflow:The default stddev value of initializer will change from \"1/sqrt(vocab_size)\" to \"1/sqrt(dimension)\" after 2017/02/25.\n",
      "WARNING:tensorflow:The default stddev value of initializer will change from \"1/sqrt(vocab_size)\" to \"1/sqrt(dimension)\" after 2017/02/25.\n",
      "WARNING:tensorflow:The default stddev value of initializer will change from \"1/sqrt(vocab_size)\" to \"1/sqrt(dimension)\" after 2017/02/25.\n",
      "wide and deep columns configured\n"
     ]
    }
   ],
   "source": [
    "# Wide columns and deep columns.\n",
    "# wide_columns = [gender, race, native_country,\n",
    "#       education, occupation, workclass,\n",
    "#       marital_status, relationship,\n",
    "#       age_buckets, education_occupation,\n",
    "#       age_race_occupation, country_occupation]\n",
    "\n",
    "# deep_columns = [\n",
    "#   tf.contrib.layers.embedding_column(workclass, dimension=8),\n",
    "#   tf.contrib.layers.embedding_column(education, dimension=8),\n",
    "#   tf.contrib.layers.embedding_column(marital_status, dimension=8),\n",
    "#   tf.contrib.layers.embedding_column(gender, dimension=8),\n",
    "#   tf.contrib.layers.embedding_column(relationship, dimension=8),\n",
    "#   tf.contrib.layers.embedding_column(race, dimension=8),\n",
    "#   tf.contrib.layers.embedding_column(native_country, dimension=8),\n",
    "#   tf.contrib.layers.embedding_column(occupation, dimension=8),\n",
    "#   age,\n",
    "#   education_num,\n",
    "#   capital_gain,\n",
    "#   capital_loss,\n",
    "#   hours_per_week,\n",
    "# ]\n",
    "\n",
    "# Embeddings for wide columns into deep columns\n",
    "for col in wide_columns:\n",
    "    deep_columns.append(tf.contrib.layers.embedding_column(col, \n",
    "                                                           dimension=8))\n",
    "\n",
    "print('wide and deep columns configured')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 构建模型\n",
    "\n",
    "你可以根据实际情况构建“宽模型”、“深模型”、“深度宽度模型”\n",
    "\n",
    "* **Wide**: 相当于逻辑回归\n",
    "* **Deep**: 相当于多层感知器\n",
    "* **Wide & Deep**: 组合两种结构\n",
    "\n",
    "这里有2个参数`hidden_units` 或者 `dnn_hidden_units`可以指定隐层的节点个数，比如`[12, 20, 15]`构建3层神经元个数分别为12、20、15的隐层。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model directory = ./models/model_WIDE_AND_DEEP_1525425429\n",
      "WARNING:tensorflow:From <ipython-input-7-e9002d0430ed>:37: calling __init__ (from tensorflow.contrib.learn.python.learn.estimators.dnn_linear_combined) with fix_global_step_increment_bug=False is deprecated and will be removed after 2017-04-15.\n",
      "Instructions for updating:\n",
      "Please set fix_global_step_increment_bug=True and update training steps in your pipeline. See pydoc for details.\n",
      "INFO:tensorflow:Using config: {'_save_checkpoints_secs': None, '_num_ps_replicas': 0, '_keep_checkpoint_max': 5, '_task_type': None, '_is_chief': True, '_cluster_spec': <tensorflow.python.training.server_lib.ClusterSpec object at 0x9280ad0>, '_model_dir': './models/model_WIDE_AND_DEEP_1525425429', '_save_checkpoints_steps': 100, '_keep_checkpoint_every_n_hours': 10000, '_session_config': None, '_tf_random_seed': None, '_save_summary_steps': 100, '_environment': 'local', '_num_worker_replicas': 0, '_task_id': 0, '_log_step_count_steps': 100, '_tf_config': gpu_options {\n",
      "  per_process_gpu_memory_fraction: 1.0\n",
      "}\n",
      ", '_evaluation_master': '', '_master': ''}\n",
      "estimator built\n"
     ]
    }
   ],
   "source": [
    "def create_model_dir(model_type):\n",
    "    # 返回类似这样的结果 models/model_WIDE_AND_DEEP_1493043407\n",
    "    return './models/model_' + model_type + '_' + str(int(time.time()))\n",
    "\n",
    "# 指定模型文件夹\n",
    "def get_model(model_type, model_dir):\n",
    "    print(\"Model directory = %s\" % model_dir)\n",
    "    \n",
    "    # 对checkpoint去做设定\n",
    "    runconfig = tf.contrib.learn.RunConfig(\n",
    "        save_checkpoints_secs=None,\n",
    "        save_checkpoints_steps = 100,\n",
    "    )\n",
    "    \n",
    "    m = None\n",
    "    \n",
    "    # 宽模型\n",
    "    if model_type == 'WIDE':\n",
    "        m = tf.contrib.learn.LinearClassifier(\n",
    "            model_dir=model_dir, \n",
    "            feature_columns=wide_columns)\n",
    "\n",
    "    # 深度模型\n",
    "    if model_type == 'DEEP':\n",
    "        m = tf.contrib.learn.DNNClassifier(\n",
    "            model_dir=model_dir,\n",
    "            feature_columns=deep_columns,\n",
    "            hidden_units=[100, 50, 25])\n",
    "\n",
    "    # 宽度深度模型\n",
    "    if model_type == 'WIDE_AND_DEEP':\n",
    "        m = tf.contrib.learn.DNNLinearCombinedClassifier(\n",
    "            model_dir=model_dir,\n",
    "            linear_feature_columns=wide_columns,\n",
    "            dnn_feature_columns=deep_columns,\n",
    "            dnn_hidden_units=[100, 70, 50, 25],\n",
    "            config=runconfig)\n",
    "        \n",
    "    print('estimator built')\n",
    "    \n",
    "    return m\n",
    "    \n",
    "\n",
    "MODEL_TYPE = 'WIDE_AND_DEEP'\n",
    "model_dir = create_model_dir(model_type=MODEL_TYPE)\n",
    "m = get_model(model_type=MODEL_TYPE, model_dir=model_dir)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 评估\n",
    "from tensorflow.contrib.learn.python.learn import evaluable\n",
    "isinstance(m, evaluable.Evaluable)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 拟合与模型训练\n",
    "\n",
    "执行`fit()`函数训练模型，可以试试不同的`train_steps`和`BATCH_SIZE`参数，会影响速度和结果"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# 训练文件与测试文件\n",
    "train_file = \"./criteo_data/criteo_train.txt\"\n",
    "eval_file  = \"./criteo_data/criteo_test.txt\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:Rank of input Tensor (1) should be the same as output_rank (2) for column. Will attempt to expand dims. It is highly recommended that you resize your input, as this behavior may change.\n",
      "WARNING:tensorflow:Rank of input Tensor (1) should be the same as output_rank (2) for column. Will attempt to expand dims. It is highly recommended that you resize your input, as this behavior may change.\n",
      "WARNING:tensorflow:Rank of input Tensor (1) should be the same as output_rank (2) for column. Will attempt to expand dims. It is highly recommended that you resize your input, as this behavior may change.\n",
      "WARNING:tensorflow:Rank of input Tensor (1) should be the same as output_rank (2) for column. Will attempt to expand dims. It is highly recommended that you resize your input, as this behavior may change.\n",
      "WARNING:tensorflow:Rank of input Tensor (1) should be the same as output_rank (2) for column. Will attempt to expand dims. It is highly recommended that you resize your input, as this behavior may change.\n",
      "WARNING:tensorflow:Rank of input Tensor (1) should be the same as output_rank (2) for column. Will attempt to expand dims. It is highly recommended that you resize your input, as this behavior may change.\n",
      "WARNING:tensorflow:Rank of input Tensor (1) should be the same as output_rank (2) for column. Will attempt to expand dims. It is highly recommended that you resize your input, as this behavior may change.\n",
      "WARNING:tensorflow:Rank of input Tensor (1) should be the same as output_rank (2) for column. Will attempt to expand dims. It is highly recommended that you resize your input, as this behavior may change.\n",
      "WARNING:tensorflow:Rank of input Tensor (1) should be the same as output_rank (2) for column. Will attempt to expand dims. It is highly recommended that you resize your input, as this behavior may change.\n",
      "WARNING:tensorflow:Rank of input Tensor (1) should be the same as output_rank (2) for column. Will attempt to expand dims. It is highly recommended that you resize your input, as this behavior may change.\n",
      "WARNING:tensorflow:Rank of input Tensor (1) should be the same as output_rank (2) for column. Will attempt to expand dims. It is highly recommended that you resize your input, as this behavior may change.\n",
      "WARNING:tensorflow:Rank of input Tensor (1) should be the same as output_rank (2) for column. Will attempt to expand dims. It is highly recommended that you resize your input, as this behavior may change.\n",
      "WARNING:tensorflow:Rank of input Tensor (1) should be the same as output_rank (2) for column. Will attempt to expand dims. It is highly recommended that you resize your input, as this behavior may change.\n",
      "WARNING:tensorflow:Casting <dtype: 'int32'> labels to bool.\n",
      "WARNING:tensorflow:Casting <dtype: 'int32'> labels to bool.\n",
      "INFO:tensorflow:Create CheckpointSaverHook.\n",
      "INFO:tensorflow:Saving checkpoints for 2 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:loss = 383.61926, step = 2\n",
      "INFO:tensorflow:Saving checkpoints for 104 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 23.1193\n",
      "INFO:tensorflow:loss = 0.52519834, step = 202 (7.250 sec)\n",
      "INFO:tensorflow:Saving checkpoints for 206 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 28.1098\n",
      "INFO:tensorflow:Saving checkpoints for 308 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 34.8379\n",
      "INFO:tensorflow:loss = 0.52586925, step = 402 (5.846 sec)\n",
      "INFO:tensorflow:Saving checkpoints for 410 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 32.676\n",
      "INFO:tensorflow:Saving checkpoints for 512 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 34.5047\n",
      "INFO:tensorflow:loss = 0.50194484, step = 602 (5.955 sec)\n",
      "INFO:tensorflow:Saving checkpoints for 614 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 34.8302\n",
      "INFO:tensorflow:Saving checkpoints for 716 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 32.3302\n",
      "INFO:tensorflow:loss = 0.53253466, step = 802 (6.003 sec)\n",
      "INFO:tensorflow:Saving checkpoints for 818 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 34.7999\n",
      "INFO:tensorflow:Saving checkpoints for 920 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 34.8958\n",
      "INFO:tensorflow:loss = 0.5117367, step = 1002 (5.747 sec)\n",
      "INFO:tensorflow:Saving checkpoints for 1022 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 32.8373\n",
      "INFO:tensorflow:Saving checkpoints for 1124 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 34.6937\n",
      "INFO:tensorflow:loss = 0.50721353, step = 1202 (5.966 sec)\n",
      "INFO:tensorflow:Saving checkpoints for 1226 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 35.1806\n",
      "INFO:tensorflow:Saving checkpoints for 1328 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 32.6754\n",
      "INFO:tensorflow:loss = 0.5044991, step = 1402 (5.916 sec)\n",
      "INFO:tensorflow:Saving checkpoints for 1430 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 34.8313\n",
      "INFO:tensorflow:Saving checkpoints for 1532 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 32.7576\n",
      "INFO:tensorflow:loss = 0.5024924, step = 1602 (5.969 sec)\n",
      "INFO:tensorflow:Saving checkpoints for 1634 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 34.8521\n",
      "INFO:tensorflow:Saving checkpoints for 1736 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 35.2324\n",
      "INFO:tensorflow:loss = 0.5241726, step = 1802 (5.717 sec)\n",
      "INFO:tensorflow:Saving checkpoints for 1838 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 33.0927\n",
      "INFO:tensorflow:Saving checkpoints for 1940 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 34.9787\n",
      "INFO:tensorflow:loss = 0.52364856, step = 2002 (5.918 sec)\n",
      "INFO:tensorflow:Saving checkpoints for 2042 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 35.3599\n",
      "INFO:tensorflow:Saving checkpoints for 2144 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 32.9638\n",
      "INFO:tensorflow:loss = 0.5026111, step = 2202 (5.891 sec)\n",
      "INFO:tensorflow:Saving checkpoints for 2246 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 35.0174\n",
      "INFO:tensorflow:Saving checkpoints for 2348 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 34.9178\n",
      "INFO:tensorflow:loss = 0.5019771, step = 2402 (5.738 sec)\n",
      "INFO:tensorflow:Saving checkpoints for 2450 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 32.9129\n",
      "INFO:tensorflow:Saving checkpoints for 2552 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 35.285\n",
      "INFO:tensorflow:loss = 0.4845492, step = 2602 (5.912 sec)\n",
      "INFO:tensorflow:Saving checkpoints for 2654 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 34.8518\n",
      "INFO:tensorflow:Saving checkpoints for 2756 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 33.0218\n",
      "INFO:tensorflow:loss = 0.51637685, step = 2802 (5.920 sec)\n",
      "INFO:tensorflow:Saving checkpoints for 2858 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 35.0541\n",
      "INFO:tensorflow:Saving checkpoints for 2960 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 32.9527\n",
      "INFO:tensorflow:loss = 0.50076705, step = 3002 (5.927 sec)\n",
      "INFO:tensorflow:Saving checkpoints for 3062 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 34.8167\n",
      "INFO:tensorflow:Saving checkpoints for 3164 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 34.7644\n",
      "INFO:tensorflow:loss = 0.49627173, step = 3202 (5.769 sec)\n",
      "INFO:tensorflow:Saving checkpoints for 3266 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 32.4713\n",
      "INFO:tensorflow:Saving checkpoints for 3368 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 34.9586\n",
      "INFO:tensorflow:loss = 0.4953499, step = 3402 (5.974 sec)\n",
      "INFO:tensorflow:Saving checkpoints for 3470 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 35.249\n",
      "INFO:tensorflow:Saving checkpoints for 3572 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 33.069\n",
      "INFO:tensorflow:loss = 0.49573877, step = 3602 (5.892 sec)\n",
      "INFO:tensorflow:Saving checkpoints for 3674 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 34.7109\n",
      "INFO:tensorflow:Saving checkpoints for 3776 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 34.9269\n",
      "INFO:tensorflow:loss = 0.5173947, step = 3802 (5.770 sec)\n",
      "INFO:tensorflow:Saving checkpoints for 3878 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 32.7293\n",
      "INFO:tensorflow:Saving checkpoints for 3980 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 35.3551\n",
      "INFO:tensorflow:loss = 0.5168913, step = 4002 (5.907 sec)\n",
      "INFO:tensorflow:Saving checkpoints for 4082 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 35.5056\n",
      "INFO:tensorflow:Saving checkpoints for 4184 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 32.5982\n",
      "INFO:tensorflow:loss = 0.4942948, step = 4202 (5.909 sec)\n",
      "INFO:tensorflow:Saving checkpoints for 4286 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 34.9633\n",
      "INFO:tensorflow:Saving checkpoints for 4388 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 35.1609\n",
      "INFO:tensorflow:loss = 0.49204993, step = 4402 (5.732 sec)\n",
      "INFO:tensorflow:Saving checkpoints for 4490 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 33.0157\n",
      "INFO:tensorflow:Saving checkpoints for 4592 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 34.5001\n",
      "INFO:tensorflow:loss = 0.47883478, step = 4602 (5.962 sec)\n",
      "INFO:tensorflow:Saving checkpoints for 4694 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 32.7373\n",
      "INFO:tensorflow:Saving checkpoints for 4796 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 34.6698\n",
      "INFO:tensorflow:loss = 0.51010066, step = 4802 (5.964 sec)\n",
      "INFO:tensorflow:Saving checkpoints for 4898 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 34.7543\n",
      "INFO:tensorflow:Saving checkpoints for 5000 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 32.8706\n",
      "INFO:tensorflow:loss = 0.496355, step = 5002 (5.952 sec)\n",
      "INFO:tensorflow:Saving checkpoints for 5102 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 35.1737\n",
      "INFO:tensorflow:loss = 0.48928428, step = 5202 (5.082 sec)\n",
      "INFO:tensorflow:Saving checkpoints for 5204 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 35.1487\n",
      "INFO:tensorflow:Saving checkpoints for 5306 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 32.471\n",
      "INFO:tensorflow:loss = 0.4903523, step = 5402 (5.954 sec)\n",
      "INFO:tensorflow:Saving checkpoints for 5408 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 34.9119\n",
      "INFO:tensorflow:Saving checkpoints for 5510 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 34.9811\n",
      "INFO:tensorflow:loss = 0.49182308, step = 5602 (5.744 sec)\n",
      "INFO:tensorflow:Saving checkpoints for 5612 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 32.6137\n",
      "INFO:tensorflow:Saving checkpoints for 5714 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 34.7582\n",
      "INFO:tensorflow:loss = 0.51414853, step = 5802 (5.969 sec)\n",
      "INFO:tensorflow:Saving checkpoints for 5816 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 32.8651\n",
      "INFO:tensorflow:Saving checkpoints for 5918 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 34.7809\n",
      "INFO:tensorflow:loss = 0.5147529, step = 6002 (5.956 sec)\n",
      "INFO:tensorflow:Saving checkpoints for 6020 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 35.0825\n",
      "INFO:tensorflow:Saving checkpoints for 6122 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 33.0911\n",
      "INFO:tensorflow:loss = 0.491095, step = 6202 (5.899 sec)\n",
      "INFO:tensorflow:Saving checkpoints for 6224 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 34.835\n",
      "INFO:tensorflow:Saving checkpoints for 6326 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 34.4511\n",
      "INFO:tensorflow:loss = 0.4875968, step = 6402 (5.800 sec)\n",
      "INFO:tensorflow:Saving checkpoints for 6428 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 33.0431\n",
      "INFO:tensorflow:Saving checkpoints for 6530 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 35.0124\n",
      "INFO:tensorflow:loss = 0.47473788, step = 6602 (5.911 sec)\n",
      "INFO:tensorflow:Saving checkpoints for 6632 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 34.8711\n",
      "INFO:tensorflow:Saving checkpoints for 6734 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 32.5408\n",
      "INFO:tensorflow:loss = 0.50613326, step = 6802 (5.974 sec)\n",
      "INFO:tensorflow:Saving checkpoints for 6836 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 34.79\n",
      "INFO:tensorflow:Saving checkpoints for 6938 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 34.9317\n",
      "INFO:tensorflow:loss = 0.49292383, step = 7002 (5.766 sec)\n",
      "INFO:tensorflow:Saving checkpoints for 7040 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 32.6444\n",
      "INFO:tensorflow:Saving checkpoints for 7142 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 34.7767\n",
      "INFO:tensorflow:loss = 0.4855942, step = 7202 (5.964 sec)\n",
      "INFO:tensorflow:Saving checkpoints for 7244 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 35.0477\n",
      "INFO:tensorflow:Saving checkpoints for 7346 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 32.8509\n",
      "INFO:tensorflow:loss = 0.4879568, step = 7402 (5.923 sec)\n",
      "INFO:tensorflow:Saving checkpoints for 7448 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 34.9138\n",
      "INFO:tensorflow:Saving checkpoints for 7550 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 34.951\n",
      "INFO:tensorflow:loss = 0.48870382, step = 7602 (5.758 sec)\n",
      "INFO:tensorflow:Saving checkpoints for 7652 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 32.7741\n",
      "INFO:tensorflow:Saving checkpoints for 7754 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 35.1165\n",
      "INFO:tensorflow:loss = 0.51130104, step = 7802 (5.925 sec)\n",
      "INFO:tensorflow:Saving checkpoints for 7856 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 34.895\n",
      "INFO:tensorflow:Saving checkpoints for 7958 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 32.9485\n",
      "INFO:tensorflow:loss = 0.5131527, step = 8002 (5.928 sec)\n",
      "INFO:tensorflow:Saving checkpoints for 8060 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 34.8513\n",
      "INFO:tensorflow:Saving checkpoints for 8162 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 33.2702\n",
      "INFO:tensorflow:loss = 0.48932356, step = 8202 (5.903 sec)\n",
      "INFO:tensorflow:Saving checkpoints for 8264 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 35.1738\n",
      "INFO:tensorflow:Saving checkpoints for 8366 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 35.1566\n",
      "INFO:tensorflow:loss = 0.48442444, step = 8402 (5.708 sec)\n",
      "INFO:tensorflow:Saving checkpoints for 8468 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 33.1651\n",
      "INFO:tensorflow:Saving checkpoints for 8570 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 35.1729\n",
      "INFO:tensorflow:loss = 0.47236505, step = 8602 (5.889 sec)\n",
      "INFO:tensorflow:Saving checkpoints for 8672 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 34.3996\n",
      "INFO:tensorflow:Saving checkpoints for 8774 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 33.2737\n",
      "INFO:tensorflow:loss = 0.5052763, step = 8802 (5.944 sec)\n",
      "INFO:tensorflow:Saving checkpoints for 8876 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 34.9456\n",
      "INFO:tensorflow:Saving checkpoints for 8978 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 34.8424\n",
      "INFO:tensorflow:loss = 0.49116126, step = 9002 (5.758 sec)\n",
      "INFO:tensorflow:Saving checkpoints for 9080 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 32.615\n",
      "INFO:tensorflow:Saving checkpoints for 9182 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 34.5405\n",
      "INFO:tensorflow:loss = 0.4835304, step = 9202 (5.983 sec)\n",
      "INFO:tensorflow:Saving checkpoints for 9284 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 32.7099\n",
      "INFO:tensorflow:Saving checkpoints for 9386 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 35.586\n",
      "INFO:tensorflow:loss = 0.4846368, step = 9402 (5.897 sec)\n",
      "INFO:tensorflow:Saving checkpoints for 9488 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 35.0763\n",
      "INFO:tensorflow:Saving checkpoints for 9590 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 32.8913\n",
      "INFO:tensorflow:loss = 0.4866239, step = 9602 (5.927 sec)\n",
      "INFO:tensorflow:Saving checkpoints for 9692 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 34.9176\n",
      "INFO:tensorflow:Saving checkpoints for 9794 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 34.8555\n",
      "INFO:tensorflow:loss = 0.51006484, step = 9802 (5.754 sec)\n",
      "INFO:tensorflow:Saving checkpoints for 9896 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 31.9199\n",
      "INFO:tensorflow:Saving checkpoints for 9998 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 35.0719\n",
      "INFO:tensorflow:loss = 0.51255614, step = 10002 (6.015 sec)\n",
      "INFO:tensorflow:Saving checkpoints for 10100 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 35.0625\n",
      "INFO:tensorflow:Saving checkpoints for 10202 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 32.8023\n",
      "INFO:tensorflow:loss = 0.48584107, step = 10202 (5.930 sec)\n",
      "INFO:tensorflow:Saving checkpoints for 10304 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 34.5218\n",
      "INFO:tensorflow:loss = 0.48140407, step = 10402 (5.125 sec)\n",
      "INFO:tensorflow:Saving checkpoints for 10406 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 35.6643\n",
      "INFO:tensorflow:Saving checkpoints for 10508 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 32.9589\n",
      "INFO:tensorflow:loss = 0.47275436, step = 10602 (5.871 sec)\n",
      "INFO:tensorflow:Saving checkpoints for 10610 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 34.9231\n",
      "INFO:tensorflow:Saving checkpoints for 10712 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 35.1866\n",
      "INFO:tensorflow:loss = 0.5033426, step = 10802 (5.726 sec)\n",
      "INFO:tensorflow:Saving checkpoints for 10814 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 32.9069\n",
      "INFO:tensorflow:Saving checkpoints for 10916 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 35.0843\n",
      "INFO:tensorflow:loss = 0.4893472, step = 11002 (5.930 sec)\n",
      "INFO:tensorflow:Saving checkpoints for 11018 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 34.8252\n",
      "INFO:tensorflow:Saving checkpoints for 11120 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 32.8465\n",
      "INFO:tensorflow:loss = 0.48183277, step = 11202 (5.936 sec)\n",
      "INFO:tensorflow:Saving checkpoints for 11222 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 35.0827\n",
      "INFO:tensorflow:Saving checkpoints for 11324 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 34.8579\n",
      "INFO:tensorflow:loss = 0.48213634, step = 11402 (5.755 sec)\n",
      "INFO:tensorflow:Saving checkpoints for 11426 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 32.733\n",
      "INFO:tensorflow:Saving checkpoints for 11528 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 35.2039\n",
      "INFO:tensorflow:loss = 0.48272803, step = 11602 (5.919 sec)\n",
      "INFO:tensorflow:Saving checkpoints for 11630 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 32.9725\n",
      "INFO:tensorflow:Saving checkpoints for 11732 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 34.789\n",
      "INFO:tensorflow:loss = 0.506504, step = 11802 (5.931 sec)\n",
      "INFO:tensorflow:Saving checkpoints for 11834 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 35.24\n",
      "INFO:tensorflow:Saving checkpoints for 11936 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 32.5431\n",
      "INFO:tensorflow:loss = 0.51041526, step = 12002 (5.935 sec)\n",
      "INFO:tensorflow:Saving checkpoints for 12038 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 35.3221\n",
      "INFO:tensorflow:Saving checkpoints for 12140 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 34.7584\n",
      "INFO:tensorflow:loss = 0.4840238, step = 12202 (5.747 sec)\n",
      "INFO:tensorflow:Saving checkpoints for 12242 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 32.6937\n",
      "INFO:tensorflow:Saving checkpoints for 12344 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 34.9127\n",
      "INFO:tensorflow:loss = 0.48007318, step = 12402 (5.950 sec)\n",
      "INFO:tensorflow:Saving checkpoints for 12446 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 35.1066\n",
      "INFO:tensorflow:Saving checkpoints for 12548 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 32.0939\n",
      "INFO:tensorflow:loss = 0.4707959, step = 12602 (6.005 sec)\n",
      "INFO:tensorflow:Saving checkpoints for 12650 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 34.3398\n",
      "INFO:tensorflow:Saving checkpoints for 12752 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 32.6643\n",
      "INFO:tensorflow:loss = 0.50073105, step = 12802 (5.998 sec)\n",
      "INFO:tensorflow:Saving checkpoints for 12854 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 35.1485\n",
      "INFO:tensorflow:Saving checkpoints for 12956 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 35.2198\n",
      "INFO:tensorflow:loss = 0.48709363, step = 13002 (5.706 sec)\n",
      "INFO:tensorflow:Saving checkpoints for 13058 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 32.6911\n",
      "INFO:tensorflow:Saving checkpoints for 13160 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 34.894\n",
      "INFO:tensorflow:loss = 0.47868958, step = 13202 (5.958 sec)\n",
      "INFO:tensorflow:Saving checkpoints for 13262 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 35.0573\n",
      "INFO:tensorflow:Saving checkpoints for 13364 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 32.7856\n",
      "INFO:tensorflow:loss = 0.4787089, step = 13402 (5.923 sec)\n",
      "INFO:tensorflow:Saving checkpoints for 13466 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 35.5471\n",
      "INFO:tensorflow:Saving checkpoints for 13568 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 35.0263\n",
      "INFO:tensorflow:loss = 0.4809025, step = 13602 (5.705 sec)\n",
      "INFO:tensorflow:Saving checkpoints for 13670 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 32.7781\n",
      "INFO:tensorflow:Saving checkpoints for 13772 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 34.9871\n",
      "INFO:tensorflow:loss = 0.50386125, step = 13802 (5.930 sec)\n",
      "INFO:tensorflow:Saving checkpoints for 13874 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 34.986\n",
      "INFO:tensorflow:Saving checkpoints for 13976 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 32.8169\n",
      "INFO:tensorflow:loss = 0.5093261, step = 14002 (5.930 sec)\n",
      "INFO:tensorflow:Saving checkpoints for 14078 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 34.668\n",
      "INFO:tensorflow:Saving checkpoints for 14180 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 35.005\n",
      "INFO:tensorflow:loss = 0.48381323, step = 14202 (5.775 sec)\n",
      "INFO:tensorflow:Saving checkpoints for 14282 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 32.7084\n",
      "INFO:tensorflow:Saving checkpoints for 14384 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 34.8872\n",
      "INFO:tensorflow:loss = 0.48138177, step = 14402 (5.949 sec)\n",
      "INFO:tensorflow:Saving checkpoints for 14486 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 35.2346\n",
      "INFO:tensorflow:Saving checkpoints for 14588 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 32.659\n",
      "INFO:tensorflow:loss = 0.4707154, step = 14602 (5.931 sec)\n",
      "INFO:tensorflow:Saving checkpoints for 14690 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 34.4108\n",
      "INFO:tensorflow:Saving checkpoints for 14792 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 34.6803\n",
      "INFO:tensorflow:loss = 0.502075, step = 14802 (5.819 sec)\n",
      "INFO:tensorflow:Saving checkpoints for 14894 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 32.757\n",
      "INFO:tensorflow:Saving checkpoints for 14996 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 34.6987\n",
      "INFO:tensorflow:loss = 0.48955604, step = 15002 (5.961 sec)\n",
      "INFO:tensorflow:Saving checkpoints for 15098 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 32.9673\n",
      "INFO:tensorflow:Saving checkpoints for 15200 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 35.2175\n",
      "INFO:tensorflow:loss = 0.4818753, step = 15202 (5.903 sec)\n",
      "INFO:tensorflow:Saving checkpoints for 15302 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 34.6938\n",
      "INFO:tensorflow:loss = 0.48233324, step = 15402 (5.121 sec)\n",
      "INFO:tensorflow:Saving checkpoints for 15404 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 32.9636\n",
      "INFO:tensorflow:Saving checkpoints for 15506 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 34.8218\n",
      "INFO:tensorflow:loss = 0.4845931, step = 15602 (5.931 sec)\n",
      "INFO:tensorflow:Saving checkpoints for 15608 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 34.4433\n",
      "INFO:tensorflow:Saving checkpoints for 15710 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 32.8916\n",
      "INFO:tensorflow:loss = 0.50622445, step = 15802 (5.968 sec)\n",
      "INFO:tensorflow:Saving checkpoints for 15812 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 35.0151\n",
      "INFO:tensorflow:Saving checkpoints for 15914 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 34.5828\n",
      "INFO:tensorflow:loss = 0.51010966, step = 16002 (5.775 sec)\n",
      "INFO:tensorflow:Saving checkpoints for 16016 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 32.9331\n",
      "INFO:tensorflow:Saving checkpoints for 16118 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 35.1875\n",
      "INFO:tensorflow:loss = 0.4836615, step = 16202 (5.902 sec)\n",
      "INFO:tensorflow:Saving checkpoints for 16220 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 32.8104\n",
      "INFO:tensorflow:Saving checkpoints for 16322 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 34.8741\n",
      "INFO:tensorflow:loss = 0.48098013, step = 16402 (5.958 sec)\n",
      "INFO:tensorflow:Saving checkpoints for 16424 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 34.8091\n",
      "INFO:tensorflow:Saving checkpoints for 16526 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 32.5486\n",
      "INFO:tensorflow:loss = 0.4695204, step = 16602 (5.969 sec)\n",
      "INFO:tensorflow:Saving checkpoints for 16628 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 35.0424\n",
      "INFO:tensorflow:Saving checkpoints for 16730 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 34.8442\n",
      "INFO:tensorflow:loss = 0.50164527, step = 16802 (5.755 sec)\n",
      "INFO:tensorflow:Saving checkpoints for 16832 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 32.7707\n",
      "INFO:tensorflow:Saving checkpoints for 16934 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 35.0926\n",
      "INFO:tensorflow:loss = 0.48918447, step = 17002 (5.921 sec)\n",
      "INFO:tensorflow:Saving checkpoints for 17036 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 35.2155\n",
      "INFO:tensorflow:Saving checkpoints for 17138 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 33.0074\n",
      "INFO:tensorflow:loss = 0.47913647, step = 17202 (5.906 sec)\n",
      "INFO:tensorflow:Saving checkpoints for 17240 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 35.1414\n",
      "INFO:tensorflow:Saving checkpoints for 17342 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 35.1936\n",
      "INFO:tensorflow:loss = 0.47959447, step = 17402 (5.718 sec)\n",
      "INFO:tensorflow:Saving checkpoints for 17444 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 32.6898\n",
      "INFO:tensorflow:Saving checkpoints for 17546 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 34.8186\n",
      "INFO:tensorflow:loss = 0.48204988, step = 17602 (5.954 sec)\n",
      "INFO:tensorflow:Saving checkpoints for 17648 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 35.1228\n",
      "INFO:tensorflow:Saving checkpoints for 17750 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 32.0315\n",
      "INFO:tensorflow:loss = 0.5047125, step = 17802 (5.994 sec)\n",
      "INFO:tensorflow:Saving checkpoints for 17852 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 35.1292\n",
      "INFO:tensorflow:Saving checkpoints for 17954 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 34.8264\n",
      "INFO:tensorflow:loss = 0.50951916, step = 18002 (5.747 sec)\n",
      "INFO:tensorflow:Saving checkpoints for 18056 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 33.1272\n",
      "INFO:tensorflow:Saving checkpoints for 18158 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 34.8218\n",
      "INFO:tensorflow:loss = 0.48249555, step = 18202 (5.922 sec)\n",
      "INFO:tensorflow:Saving checkpoints for 18260 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 34.8553\n",
      "INFO:tensorflow:Saving checkpoints for 18362 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 32.9885\n",
      "INFO:tensorflow:loss = 0.47924712, step = 18402 (5.926 sec)\n",
      "INFO:tensorflow:Saving checkpoints for 18464 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 34.8848\n",
      "INFO:tensorflow:Saving checkpoints for 18566 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 33.395\n",
      "INFO:tensorflow:loss = 0.46886802, step = 18602 (5.893 sec)\n",
      "INFO:tensorflow:Saving checkpoints for 18668 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 35.2775\n",
      "INFO:tensorflow:Saving checkpoints for 18770 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 35.5929\n",
      "INFO:tensorflow:loss = 0.5005961, step = 18802 (5.667 sec)\n",
      "INFO:tensorflow:Saving checkpoints for 18872 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 32.9238\n",
      "INFO:tensorflow:Saving checkpoints for 18974 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 35.0456\n",
      "INFO:tensorflow:loss = 0.4847619, step = 19002 (5.913 sec)\n",
      "INFO:tensorflow:Saving checkpoints for 19076 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 35.0182\n",
      "INFO:tensorflow:Saving checkpoints for 19178 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 32.9734\n",
      "INFO:tensorflow:loss = 0.47770432, step = 19202 (5.922 sec)\n",
      "INFO:tensorflow:Saving checkpoints for 19280 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 35.4214\n",
      "INFO:tensorflow:Saving checkpoints for 19382 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 34.6074\n",
      "INFO:tensorflow:loss = 0.4758588, step = 19402 (5.731 sec)\n",
      "INFO:tensorflow:Saving checkpoints for 19484 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 32.7778\n",
      "INFO:tensorflow:Saving checkpoints for 19586 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 35.1873\n",
      "INFO:tensorflow:loss = 0.48031697, step = 19602 (5.924 sec)\n",
      "INFO:tensorflow:Saving checkpoints for 19688 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 33.0876\n",
      "INFO:tensorflow:Saving checkpoints for 19790 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 35.0324\n",
      "INFO:tensorflow:loss = 0.5011429, step = 19802 (5.911 sec)\n",
      "INFO:tensorflow:Saving checkpoints for 19892 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 34.3502\n",
      "INFO:tensorflow:Saving checkpoints for 19994 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:global_step/sec: 32.73\n",
      "INFO:tensorflow:loss = 0.50794, step = 20002 (5.994 sec)\n",
      "INFO:tensorflow:Saving checkpoints for 20002 into ./models/model_WIDE_AND_DEEP_1525425429/model.ckpt.\n",
      "INFO:tensorflow:Loss for final step: 0.50794.\n",
      "fit done\n"
     ]
    }
   ],
   "source": [
    "# This can be found with\n",
    "# wc -l train.csv\n",
    "train_sample_size = 2000000\n",
    "train_steps = train_sample_size/BATCH_SIZE*20\n",
    "\n",
    "m.fit(input_fn=generate_input_fn(train_file, BATCH_SIZE), steps=train_steps)\n",
    "\n",
    "print('fit done')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 评估模型准确率\n",
    "评估准确率"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "eval_sample_size = 500000 # this can be found with a 'wc -l eval.csv'\n",
    "eval_steps = eval_sample_size/BATCH_SIZE\n",
    "\n",
    "results = m.evaluate(input_fn=generate_input_fn(eval_file), \n",
    "                     steps=eval_steps)\n",
    "print('evaluate done')\n",
    "\n",
    "print('Accuracy: %s' % results['accuracy'])\n",
    "print(results)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "进行预估"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def pred_fn():\n",
    "    sample = [ 0, 127, 1, 3, 1683, 19, 26, 17, 475, 0, 9, 0, 3, \"05db9164\", \"8947f767\", \"11c9d79e\", \"52a787c8\", \"4cf72387\", \"fbad5c96\", \"18671b18\", \"0b153874\", \"a73ee510\", \"ceb10289\", \"77212bd7\", \"79507c6b\", \"7203f04e\", \"07d13a8f\", \"2c14c412\", \"49013ffe\", \"8efede7f\", \"bd17c3da\", \"f6a3e43b\", \"a458ea53\", \"35cd95c9\", \"ad3062eb\", \"c7dc6720\", \"3fdb382b\", \"010f6491\", \"49d68486\"]\n",
    "    sample_dict = dict(zip(FEATURE_COLUMNS, sample))\n",
    "    \n",
    "    for feature_name in CATEGORICAL_COLUMNS:\n",
    "        sample_dict[feature_name] = tf.expand_dims(sample_dict[feature_name], -1)\n",
    "        \n",
    "    for feature_name in CONTINUOUS_COLUMNS:\n",
    "        sample_dict[feature_name] = tf.constant(sample_dict[feature_name], dtype=tf.int32)\n",
    "    print(sample_dict)\n",
    "\n",
    "    return sample_dict\n",
    "\n",
    "m.predict(input_fn=pred_fn)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.14"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
